Method and apparatus for photograph finding

ABSTRACT

Digital image data including discrete photographic images of a variety of different subjects, times, and so forth, are collected and analyzed to identify specific features in the photographs. In an embodiment of the invention, distinctive markers are distributed to aid in the identification of particular subject matter. Facial recognition may also be employed. The digital image data is maintained in a database and quarried in response to search requests. The search requests include criteria specifying any feature category or other identifying information, such as date, time, and location that each photograph was taken, associated with each photograph. Candidate images are provided for review by requesters, who may select desired images for purchase or downloading.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.17/020,671, filed Sep. 14, 2020 (now U.S. Pat. No. 11,061,971), which isa continuation of U.S. application Ser. No. 16/105,876, filed Aug. 20,2018 (now U.S. Pat. No. 10,776,430), which is a continuation of U.S.application Ser. No. 15/481,595, filed Apr. 7, 2017 (now U.S. Pat. No.10,055,497), which is a continuation of U.S. application Ser. No.14/518,655, filed Oct. 20, 2014 (now U.S. Pat. No. 9,619,486), which isa continuation of applicaton Ser. No. 13/776,463, filed Feb. 25, 2013(now U.S. Pat. No. 8,867,798), which is a continuation of U.S.application Ser. No. 13/090,026, filed Apr. 19, 2011 (now U.S. Pat. No.8,385,691), which is a continuation of U.S. application Ser. No.12/874,929, filed Sep. 2, 2010 now U.S. Pat. No. 7,929,810), which is acontinuation of U.S. application Ser. No. 12/325,589 filed Dec. 1, 2008(now U.S. Pat. No. 7,844,141), which is a continuation of U.S.application Ser. No. 11/056,699, filed Feb. 10, 2005 (now Pat. No.7,460,7371 which claims priority to U.S. Provisional Appl. No.60/544,570, filed Feb. 12, 2004 the disclosures of each of theabove-referenced applications are incorporated by reference herein intheir entireties.

BACKGROUND OF THE INVENTION Field of the Invention

Photography has transformed how people conceive of the world.Photographs allow people to see all sorts of things that are actuallymany miles away and/or years preceding. Photography lets people capturemoments in time and preserve them for years to come.

Often people at a public place notice that a stranger has taken aphotograph of which they would love to have a copy, Alternatively, aftergoing somewhere, a person may bemoan the fact that he did not have aphotograph of the event (in the present context, photograph alsoincludes video, audio, or other representation).

A need exists, therefore, to provide a method and apparatus foridentifying and connecting people with photographs they want. Inaddition, there is a need to provide a method and apparatus forcharacterizing errant photographs stored on computer databases thatmakes use of a variety of attributes to reliably characterizephotographs in such a way as to reduce the amount of manual reviewnecessary to identify and connect people with the photographs they want.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

Specifically, in one embodiment, a web site is created with a databasebackend. The database is seeded with information provided by (1) thephotographer; (2) recovering metadata from the photograph; (3) readingdevices such as a Global Positioning System (GPS) device; (4)referencing the account data of the photographer (i.e., account number,photographer's zip code or area code, etc.); (5) analyzing thephotograph (i.e., computer recognizes eye color, optical characterrecognizes any text found in the photograph, recognizes the number ofpersons, the gender of persons, the hair color, the time of day byoptical character recognizing any clocks in the photograph or analyzingthe lighting conditions, the weather, etc.); (6) photograph qualityinformation; and/or (7) any other information.

A user looking for a photograph would visit the web site and search forcertain criteria. The user is then provided with a gallery of thumbnailsthat match the criteria. When the user identifies a photograph he wantsto own, he can then download the full quality version, or orderprint(s). In a preferred implementation, the user is charged some amountof money that is split between the site owner and the photographer.Alternatively, the user may be charged in some other way, such as byviewing advertisements or by exchanging credits for downloads or by someother payment or a combination thereof.

A more complete understanding of the present invention will be affordedto those skilled in the art, as well as a realization of additionaladvantages and objects thereof, by a consideration of the followingdetailed description of the preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram showing exemplary steps of a method accordingto the invention.

FIG. 2 is a diagram showing an exemplary distinctive marker forphotographic data.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

FIG. 1 illustrates exemplary steps of a method 100 according to theinvention. At optional step 102, distinctive markers may be distributedto persons desiring to contribute photographic images to a database. Themarkers may comprise, for example, distinctive bins, badges, or stickersfor placing on objects to be photographed. The markers should bedesigned so as to be easily recognized using automatic recognitionalgorithms, but should not be too conspicuous.

At step 104, image data is collected from a variety of sources. It maybe desirable to accept material from as many sources as possible, toincrease the number of images available for browsing. Optionally, imagesmay be accepted from qualified sources only.

At step 104, source information regarding each photograph is collected.For example, at step 108, the image provider may be asked forinformation, such as the time and date of the photograph, the subjectmatter, location, photographer, etc. Image metadata may also be read toobtain such information. Other ways of associating images to identifyinginformation may include, for example, reading devices such as a GlobalPositioning System (GPS) device attached to a camera or other imagecapture device, or by referencing account data of the image contributor(e.g., account number, photographer's zip code or area code, etc.).

At step 110, image data is analyzed to identify any characteristics thatmay be of interest to users. Such characteristics may include, forexample, eye color, words and sentences, a number or gender of persons,the hair color, time of day, lighting conditions, and so forth. Forfurther example, at step 112, a facial recognition program as known inthe art may be used to analyze any faces appearing in the photos at asufficiently high resolution. At step 114, the images may be analyzedfor the presence of any known markers. And at step 116, other featuresand qualities of the image may be classified, for example, whether it istaken indoors or outdoors, whether it contains people, dogs, cats, orother animals, whether it contains automobiles, airplanes, or otherobjects, and so forth. At step 118, selected feature information andother source information is associated with each image and provided toany suitable relational database.

At step 120, requests specifying search criteria for photographic imagesare received. For example, a user looking for a photograph may visit aweb site hosted by the system and fill out a search form specifyingsearch criteria of interest. The criteria may include specific subjectmatter, times, dates, and locations. For example, “Disneyland ANDMatterhorn AND blue eye AND child AND Jan. 1, 2004 AND morning” wouldsearch for a photograph or photographs taken at Disneyland's Matterhornwith a child who has blue eyes on the morning of Jan. 1, 2004.

At step 122, the image database is queried as known in the art, toidentify images that at least partially match the search criteria. Suchimages may be presented, at step 124, to the user. For example, the usermay be provided with a gallery of “thumbnail” (reduced-size) imagesgenerated from images that match the criteria. When the user identifiesa photograph he wants to own, he can then download the full qualityversion, or order print(s). In a preferred implementation, the user ischarged some amount of money that is split between the site owner andthe photographer. Alternatively, the user may be charged in some otherways such as by viewing advertisements or by exchanging credits fordownloads or by some other payment or a combination thereof. The pricecan be on a sliding scale depending on the quality of the photographthat the user downloads or the size or quality of the print. Forexample, a photograph may cost $1.00 for 1024×768 resolution or $2.00for 1600×1200 resolution. Similarly, a print may cost $1.00 for 3×5 or$5 for 8×10. For downloads, an “upgrade” may be possible by charging thedifference between the resolutions. An automated process may be used toreduce the number of pixels for purposes of having a lower qualityversion to sell.

In addition, a surcharge may be applied (even if no surcharge isrequired) for various enhancements to the photograph, such as“upconverting” to a higher resolution, eliminating red-eye, enhancingshadow, color, or brightness, etc.

Moreover, when a photographer takes photographs, he can be provided withprinted cards bearing a Uniform Resource Locator (URL) and a unique codein order that the user would be able to enter into the web site to findthe photograph or the series of photographs then being taken. Thephotographer can also distribute cards (the printed cards bearing theURL and the unique code or any other cards known to those skilled in theart) to people whom he photographs, whether intentionally orinadvertently. The photographer can further advertise the same (e.g.,the URL and the unique code) via a mark on his camera, a T-shirt, orother means.

Fixed-place cameras can also serve this function (e.g., the ofphotographer). For example, a camera set up at an intersection inHollywood might take and upload one photograph every 10 seconds.

Photographers can also be given accounts and be allowed to uploadphotographs to the site. The database is populated during this process,although additional database information can be added later by web siteusers. In addition, the number of times the photograph has beenpurchased and/or viewed can be a part of the database.

In one embodiment, the method and apparatus of the present inventionshould be capable of face recognition. It should assign values tovarious factors (i.e., ratio of distance between pupils to distance totip of nose, etc.). It would add this information to the database foruploaded photographs. A user can then upload a photograph of the targetperson and the system would then generate the same data from thatphotograph and use it to limit the possible search targets.

A provider of the present method and apparatus or a photographer canalso hand out pins, clothing, or other materials that are marked in away that allows a computer to later recognize them in a photograph. FIG.2 shows an exemplary distinctive marker 200 having an optical code 202,such as a bar code. The marker may have a color combination, distinctiveshape, lettering, bar code, or other optical pattern, or somecombination of the foregoing, that is unique to the marker. The markermay be computer generated, for example, and produced using an end-user'slaser or ink-jet printer. The marker may be associated with specificinformation, for example, a particular user account, photographer,subject matter type, person, event, or location. Users can later searchfor photographs containing an image of the marker.

Numerous distribution mechanisms exist whereby photographs may bedistributed from a source over a wide area network, such as theInternet. In some cases, the photographs are distributed using acentralized server system (such as Napster 2.0, eBay, or from a website). In other cases, the photographs are distributed using adecentralized system (such as Gnutella). In a preferred implementation,the photographs are distributed to a person using the centralized serversystem or using a central hub.

Embodiments of the present invention operate in accordance with at leastone web-hosting mechanism and a plurality of user mechanismscommunicating over a wide area network, such as the Internet.Specifically, a web-hosting mechanism includes a database, an interfaceapplication and a server, wherein the server is adapted to communicatewith a plurality of user mechanisms over a wide area network. It shouldbe appreciated that the mechanisms described can include, but are notlimited to, personal computers, mainframe computers, personal digitalassistances, wireless communication devices and all other physical andwireless connected network devices generally known to those skilled inthe art. It should further be understood that the database depicted caninclude, but is not limited, to RAM, cache memory, flash memory,magnetic disks, optical disks, removable disks, SCSI disks, IDE harddrives, tape drives, and all other types of data storage devices (andcombinations thereof, such as RAID devices) generally known to thoseskilled in the art. In addition, the mechanisms described above are forpurposes of example only and the invention is not limited thereby.

Having thus described several embodiments for photograph finding, itshould be apparent to those skilled in the art that certain advantagesof the system have been achieved. It should also be appreciated thatvarious modifications, adaptations, and alternative embodiments thereofmay be made within the scope and spirit of the present invention. Forexample, in the context of the present invention a photograph caninclude video, audio, and/or other representation of how people conceiveof the world. The invention is defined by the following claims.

What is claimed is:
 1. A method, comprising: a computer system receivinga search request that includes a digital image; the computer systemdetermining a set of multiple characteristics of the digital image,including: one or more objects appearing in the digital image; one ormore colors appearing in the digital image; and lighting conditionsappearing in the digital image; the computer system comparing the set ofmultiple characteristics with corresponding characteristics forrespective ones of a plurality of stored digital images; and thecomputer system transmitting a response to the search request, whereinthe response indicates one or more stored digital images that meet oneor more matching criteria for the comparing.
 2. The method of claim 1,wherein the set of multiple characteristics further includes one or morecharacteristics of the following characteristics: faces recognized inthe digital image; time of day represented in the digital image; adevice used to capture the digital image; and a geographic location atwhich the digital image was captured.
 3. The method of claim 1, whereinthe one or more objects include visual facial characteristics.
 4. Themethod of claim 1, wherein the one or more objects include a knownmarker obj ect.
 5. The method of claim 1, wherein at least one of themultiple characteristics is determined based on image metadata and atleast one of the multiple characteristics is determined based onanalyzing image data of the digital image.
 6. The method of claim 1,wherein the search request is received via a wide area network, thedigital image is uploaded via the wide area network, and the response istransmitted via the wide area network.
 7. The method of claim 6, whereinthe transmitting the response includes causing reduced-size versions, ofthe one or more stored digital images that meet one or more matchingcriteria, to be displayed via a web site.
 8. The method of claim 1,further comprising: the computer system receiving one or more searchparameters based on user input; wherein the comparing includes comparingcharacteristics for stored digital images with the received one or moresearch parameters.
 9. A system, comprising: one or more processors; andone or more memories having program instructions stored thereon that areexecutable by the one or more processors to: receive a search requestthat includes a digital image; determine a set of multiplecharacteristics of the digital image, including: one or more objectsappearing in the digital image; one or more colors appearing in thedigital image; and lighting conditions appearing in the digital image;compare the set of multiple characteristics with correspondingcharacteristics for respective ones of a plurality of stored digitalimages; and transmit a response to the search request, wherein theresponse indicates one or more stored digital images that meet one ormore matching criteria for the comparison.
 10. The system of claim 9,wherein the set of multiple characteristics further includes thefollowing characteristics: faces recognized in the digital image; andtime of day represented in the digital image.
 11. The system of claim10, wherein the set of multiple characteristics further includes thefollowing characteristics: a device used to capture the digital image;and a geographic location at which the digital image was captured. 12.The system of claim 9, wherein at least one of the multiplecharacteristics is determined based on image metadata and at least oneof the multiple characteristics is determined based on an analysis ofpixel data of the digital image.
 13. The system of claim 9, wherein thesearch request is received via a wide area network, the digital image isuploaded via the wide area network, and the response is transmitted viathe wide area network.
 14. A non-transitory computer-readable mediumhaving instructions stored thereon that are executable by a computingdevice to perform operations comprising: receiving a search request thatincludes a digital image; determining a set of multiple characteristicsof the digital image, including: one or more objects appearing in thedigital image; one or more colors appearing in the digital image; andlighting conditions appearing in the digital image; comparing the set ofmultiple characteristics with corresponding characteristics forrespective ones of a plurality of stored digital images; andtransmitting a response to the search request, wherein the responseindicates one or more stored digital images that meet one or morematching criteria for the comparing.
 15. The non-transitorycomputer-readable medium of claim 14, wherein the set of multiplecharacteristics further includes one or more characteristics of thefollowing characteristics: faces recognized in the digital image; timeof day represented in the digital image; a device used to capture thedigital image; and a geographic location at which the digital image wascaptured.
 16. The non-transitory computer-readable medium of claim 14,wherein at least one of the multiple characteristics is determined basedon image metadata and at least one of the multiple characteristics isdetermined based on analyzing image data of the digital image.
 17. Thenon-transitory computer-readable medium of claim 14, wherein the searchrequest is received via a wide area network, the digital image isuploaded via the wide area network, and the response is transmitted viathe wide area network.
 18. The non-transitory computer-readable mediumof claim 14, wherein the transmitting the response includes causingreduced-size versions, of the one or more stored digital images thatmeet one or more matching criteria, to be displayed via a web site. 19.The non-transitory computer-readable medium of claim 14, wherein theoperations further comprise: receiving one or more search parametersbased on user input; wherein the comparing includes comparingcharacteristics for stored digital images with the received one or moresearch parameters.
 20. The non-transitory computer-readable medium ofclaim 14, wherein the one or more objects include visual facialcharacteristics.